Aaditya Ramdas

Associate Professor Carnegie Mellon University Department of Statistics and Data Science (75%) Machine Learning Department (25%) Visiting academic, Amazon (20%). 132H Baker Hall aramdas AT {empty or stat or cs} DOT cmu FULLSTOP edu [http://www.stat.cmu.edu/~aramdas]

Biography

Aaditya Ramdas (PhD, 2015) is an Associate Professor at Carnegie Mellon University, in the Departments of Statistics and Machine Learning. He was a postdoc at UC Berkeley (2015–2018) mentored by Michael Jordan and Martin Wainwright, and obtained his PhD at CMU (2010–2015) under Aarti Singh and Larry Wasserman, receiving the Umesh K. Gavaskar Memorial Thesis Award. His undergraduate degree was in Computer Science from IIT Bombay (2005-09).

Aaditya received the Sloan fellowship in mathematics (2024), the IMS Peter Gavin Hall Early Career Prize (2023), the inaugural COPSS Emerging Leader Award (2021), and the Bernoulli New Researcher Award (2021). His work is supported by an NSF CAREER Award (2020), an Adobe Faculty Research Award (2019) and a Google Research Scholar award (2022). He was a CUSO lecturer in 2022, a Lunteren lecturer in 2023, and a keynote speaker at AISTATS 2024. His group's work has received multiple paper awards, including discussion papers at JASA and JRSSB.

Aaditya's research in mathematical statistics and learning has an eye towards designing algorithms that both have strong theoretical guarantees and also work well in practice. His main interests include post-selection inference (multiple testing, simultaneous inference), game-theoretic statistics (e-values, confidence sequences) and predictive uncertainty quantification (conformal prediction, calibration). His areas of applied interest include privacy, neuroscience, genetics and auditing (elections, real-estate, financial, fairness).

He is one of the organizers of the amazing and diverse StatML Group at CMU. Outside of work, some easy topics for conversation include travel/outdoors, trash-free living, completing the Ironman triathlon and long-distance bicycle rides. Here are some of the books he has read recently.

Curriculum Vitae

New: The E-book (an introduction to hypothesis testing with e-values)

Group

Courses, Workshops, Tutorials, Software, Talks, etc.

Advice for PhD students and assistant professors.

These keywords quickly get my attention:

• e-values and confidence sequences (e-processes, supermartingales, testing by betting, sequential inference, optional stopping, peeking and p-hacking, change detection, anytime p-values, Ville's inequality, game-theoretic statistics)

• conformal prediction and calibration (distribution-free inference, uncertainty quantification for black-box machine learning, covariate/label shift, beyond exchangeability)

• multiple testing and post-selection inference (false discovery rate, inference after model selection, online or interactive or bandit testing, post-hoc simultaneous inference)

• high-dimensional, nonparametric statistics and machine learning (kernel methods, minimax rates, dimension-agnostic inference, universal inference, differential privacy, optimization)

I work on “practical theory”, meaning that the vast majority of my papers are about designing theoretically principled algorithms that directly solve practical problems, and are usually based on simple, aesthetically elegant (in my opinion) ideas. A theoretician's goal is not to prove theorems, just as a writer's goal is not to write sentences. My goals are to improve my own (and eventually the field's) understanding of important problems, design creative algorithms for unsolved questions and figure out when and why they work (or don't), and often simply to ask an intriguing question that has not yet been asked.

News

I'm representing the National Academy of Sciences of the US in a joint meeting with the National Academy of Sciences of China (Nov'24, co-sponsored by the Kavli Foundation).

I'm co-editing a special issue on Conformal Prediction, Probabilistic Calibration and Distribution-Free Uncertainty Quantification (first round reviews complete).

I co-edited (with P. Grunwald) a special issue on Game-theoretic statistics and safe, anytime-valid inference, publishing date: May'24.

I am co-organizing a week-long workshop on game-theoretic statistics and sequential anytime-valid inference at BIRS Chennai (Jun 29 -- Jul 3, 2025). I previously co-organized a week-long workshop on game-theoretic statistics and sequential anytime-valid inference at Oberwolfach (May'24), and an earlier week-long workshop at EURANDOM (Jun'22). I've previously co-organized day-long workshops on conformal prediction and calibration: DFUQ'22 and DFUQ'21, amongst others.

I like designing new classes: I recently taught a course on game-theoretic probability, statistics and learning to 25 PhD students. I've taught other new PhD courses on multiple testing, sequential analysis, statistical machine learning, the history of machine learning, etc. (see misc)

I recently taught a 6-hour tutorial on game-theoretic statistics at GeorgiaTech, this is the first video. I also taught a 15-hour mini-course on the topic at Columbia University.

On this page, I maintain a list of companies that use our anytime-valid inference methods in publicly deployed software or products.

Preprints (under review or unsubmitted)

1. Hypothesis testing with e-values (with R. Wang).       arXiv | TLDR

2. Improving the (approximate) sequential probability ratio test by avoiding overshoot (with L. Fischer).       arXiv | TLDR

3. Conformalized interactive imitation learning: handling expert shift and intermittent feedback (with M. Zhao, R. Simmons, H. Admoni, A. Bajcsy) arXiv | website

4. β-calibration of Language Model Confidence Scores for Generative QA (with P. Manggala, A. Mastakouri, E. Kirschbaum, S. Kasiviswanathan) arXiv

5. Compound e-values and Empirical Bayes (with N. Ignatiadis, R. Wang) arXiv

6. Sequential Kernelized Stein Discrepancy (with D. Taboada).       arXiv | TLDR

7. Empirical Bernstein in smooth Banach spaces (with D. Taboada).       arXiv | TLDR

8. Robust likelihood ratio tests for composite nulls and alternatives (with A. Saha).       arXiv | TLDR

9. Practical maximally flexible sampling designs for experiments based on e-values (with A. Ly, U. Boehm, P. Grünwald, D. van Ravenzwaaij).       psyarXiv | TLDR

10. Anytime-valid inference for double/debiased machine learning of causal parameters (with A. Dalal, P. Blobaum, S. Kasiviswanathan).       arXiv | TLDR

11. Matrix concentration: order versus anti-order (with R. Malekian).       arXiv | TLDR

12. An online generalization of the (e-)Benjamini-Hochberg procedure (with L. Fischer, Z. Xu).       arXiv | TLDR

13. Online closed testing with e-values (with L. Fischer).       arXiv | TLDR

14. Multiple testing with anytime-valid Monte-Carlo p-values (with L. Fischer).       arXiv | TLDR

15. Combining exchangeable p-values (with M. Gasparin, R. Wang).       arXiv | TLDR

16. Conformal online model aggregation (with M. Gasparin).       arXiv | TLDR

17. The numeraire e-variable and reverse information projection (with M. Larsson, J. Ruf).       arXiv | TLDR

18. Combining evidence across filtrations using adjusters (with Y.J. Choe).       arXiv | TLDR | code | slides

    | blog

19. Distribution-uniform strong laws of large numbers (with I. Waudby-Smith, M. Larsson).       arXiv | TLDR

20. Positive semidefinite supermartingales and randomized matrix concentration inequalities (with H. Wang).       arXiv | TLDR

21. Testing by betting while borrowing and bargaining (with H. Wang).       arXiv | TLDR

22. Merging uncertainty sets via majority vote (with M. Gasparin).       arXiv | TLDR

23. Sequential Monte-Carlo testing by betting (with L. Fischer).       arXiv | TLDR

24. Time-uniform confidence spheres for means of random vectors (with B. Chugg, H. Wang).       arXiv | TLDR

    We use PAC-Bayesian techniques to derive multivariate confidence sequences for means, in both light and heavy tailed settings, including a multivariate empirical-Bernstein inequality for bounded random variables.

25. Distribution-uniform anytime-valid inference (with I. Waudby-Smith).       arXiv | TLDR

    We give the first distribution-uniform guarantees for asymptotic anytime-valid procedures by deriving a novel and uniformly valid strong Gaussian approximation theorem.

26. Time-uniform self-normalized concentration for vector-valued processes (with J. Whitehouse, S. Wu).       arXiv | TLDR

    We derive new self-normalized confidence sequences for multivariate means under a variety of sub-psi tail conditions, including a new multivariate empirical-Bernstein inequality for bounded observations.

27. Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance (with H. Wang).       arXiv | TLDR

    We derive and compare various one-sided and two-sided anytime-valid t-tests and analyze the width of the corresponding confidence sequences, including pointing out that an improper/flat mixture approach of Lai is rather suboptimal in practice.

28. On the near-optimality of betting confidence sets for bounded means (with S. Shekhar).       arXiv | TLDR

    We derive information theoretic lower bounds on the width of any confidence interval or confidence sequence for the mean of a bounded random variable, and show that recent betting-based confidence sets essentially achieve that lower bound.

29. Scalable causal structure learning via amortized conditional independence testing (with J. Leiner, B. Manzo, W. Tansey).       arXiv | code | TLDR

    We develop a method that leverages discrete optimization techniques for efficient causal discovery, and apply it to a cancer dataset to reveal connections between somatic gene mutations and metastases to different tissues.

30. More powerful multiple testing under dependence via randomization (with Z. Xu).       arXiv | TLDR

    We show that the power of three procedures for multiple testing under arbitrary dependence --- Benjamini-Yekutieli, e-BH and Simes --- can be strictly improved using a simple uniform external randomization.

31. A sequential test for log-concavity (with A. Gangrade, A. Rinaldo).       arXiv

32. Admissible anytime-valid sequential inference must rely on nonnegative martingales (with J. Ruf, M. Larsson, W. Koolen).       arXiv

Published (or accepted) papers

1. The extended Ville's inequality for nonintegrable nonnegative supermartingales (with H. Wang), Bernoulli, 2025.       arXiv | TLDR

   We present a rigorous study of supermartingales and their maximal inequalities without the traditional L1-integrability assumption, which typically arise as infinite mixtures of traditional, L1-integrable martingales; applications include deriving "frequentist" confidence sequences despite using Bayesian flat, non-informative priors.

2. Bias detection via signaling (with T. Lin, I. Shapira, Y. Chen, A. Procaccia), Conf. on Neural Information Processing Systems (NeurIPS), 2024.       arXiv | TLDR

3. On the existence of powerful p-values and e-values for composite hypotheses (with Z. Zhang, R. Wang), Annals of Stat., 2024.       arXiv

4. Time-uniform central limit theory and asymptotic confidence sequences (with I. Waudby-Smith, D. Arbour, R. Sinha, E. H. Kennedy), Annals of Stat., 2024.       arXiv | code

5. Randomized and exchangeable improvements of Markov's, Chebyshev's and Chernoff's inequalities (with T. Manole), Statistical Science, 2024.       arXiv

6. Post-selection inference for e-value based confidence intervals (with Z. Xu, R. Wang), Elec J. Stat., 2024.       arXiv | proc | talk | slides | TLDR

   We propose the e-BY procedure for post-selection inference (false coverage rate of chosen confidence intervals), which allows the Benjamini-Yekutieli procedure be applied under both arbitrary selection rules and dependence without an extra log factor correction when using e-value based CIs.

7. Interactive identification of individuals with positive treatment effect while controlling false discoveries (with B. Duan, L. Wasserman), J. of Causal Inference, 2024.       arXiv | proc

8. Multiple testing under negative dependence (with Z. Chi, R. Wang), Bernoulli, 2024.       arXiv

9. Total variation floodgate for variable importance inference in classification (with W. Wang, L. Janson, L. Lei), Intl. Conf. on Machine Learning (ICML), 2024.       arXiv | TLDR

   We define the expected total variation (ETV), which is an intuitive, model-agnostic measure of variable importance and design algorithms for statistical inference on the ETV under design-based/model-X assumptions.

10. Reducing sequential change detection to sequential estimation (with S. Shekhar), Intl. Conf. on Machine Learning (ICML), 2024.       arXiv | TLDR

    We reduce sequential change detection to sequential estimation using confidence sequences, much like Lorden's classical reduction to sequential testing.

11. Universal inference meets random projections: a scalable test for log-concavity (with R. Dunn, A. Gangrade, L. Wasserman), J Comp & Graphical Stat, 2024.       arXiv | code | proc | TLDR

    We develop the first provably valid test for log-concavity, which is a shape constraint for density estimation with applications across economics, survival modeling, and reliability theory. The test is based on universal inference and a scalable variant using random projections is developed.

12. De Finetti's Theorem and related results for infinite weighted exchangeable sequences (with R. Barber, E. Candes, R. Tibshirani), Bernoulli, 2024       arXiv | proc

13. Semiparametric efficient inference in adaptive experiments (with T. Cook, A. Mishler), Conference on Causal Learning and Reasoning (CLeaR), 2024.       arXiv | TLDR

    We show how to achieve semiparametric efficient confidence intervals and confidence sequences for the average treatment effect when performing adaptive experimentation, based on the Adaptive AIPW (A2IPW) estimator of Kato.

14. Anytime-valid off-policy inference for contextual bandits (with I. Waudby-Smith, L. Wu, N. Karampatziakis, P. Mineiro), ACM/IMS J of Data Science, 2024.       arXiv | proc

15. Testing exchangeability by pairwise betting (with A. Saha), Intl. Conf. on AI and Statistics (AISTATS), 2024. (oral talk)       arXiv | proc | poster | TLDR

    As an alternative to universal inference and conformal prediction, we propose a new method for testing exchangeability which uses betting on pairs of observations, which applies to general observation spaces and can be shown to be consistent against reasonable alternatives.

16. Graph fission and cross-validation (with J. Leiner), Intl. Conf. on AI and Statistics (AISTATS), 2024       arXiv | TLDR

    We extend data fission/thinning to the graphical setting to develop a procedure that takes in an input graph with noisy observations at each node, and creates multiple independent synthetic copies of the graph with the same node+edge set as the original and noisier observations at each node. We then show how this method can be used for cross-validation and structural trend estimation on graph data.

17. Online multiple testing with e-values (with Z. Xu), Intl. Conf. on AI and Statistics (AISTATS), 2024.       arXiv | TLDR

    We design an extension of the LOND algorithm for online FDR and FCR control with e-values.

18. Deep anytime-valid hypothesis testing (with T. Pandeva, P. Forré, S. Shekhar), Intl. Conf. on AI and Statistics (AISTATS), 2024.       arXiv

19. Differentially private conditional independence testing (with I. Kalemaj, S. Kasiviswanathan), Intl. Conf. on AI and Statistics (AISTATS), 2024.       arXiv | TLDR

    We derive differentially private versions of the generalized covariance measure (GCM), in both observational settings and under the model-X assumption, the primary difficulty of which arises from the fact that changing one datapoint changes all the GCM residuals.

20. E-detectors: a nonparametric framework for online changepoint detection (with J. Shin, A. Rinaldo), New England J of Stat. and Data Science, 2023.       arXiv | proc

21. A unified recipe for deriving (time-uniform) PAC-Bayes bounds (with B. Chugg, H. Wang), J of ML Research, 2023.       arXiv | proc

22. A permutation-free kernel independence test (with S. Shekhar, I. Kim), J of ML Research, 2023.       arXiv | code | proc | TLDR

    We propose a new kernel-HSIC statistic that drops half the terms of the original, and show that it has a standard normal limiting null distribution under low and high dimensional regimes, resulting in a test that is trivial to calibrate without permutations. The analysis of this statistic requires generalizing the techniques we developed earlier for the corresponding kernel-MMD statistic, to deal with the more complicated dependence structure.

23. Data fission: splitting a single data point (with J. Leiner, B. Duan, L. Wasserman), J of American Stat Assoc, 2023 arXiv | proc | poster | slides | code | talk | TLDR (Discussion paper)

    We devise an alternative to data splitting using external randomization called data fission that more efficiently splits information in many circumstances and then apply it to several examples in post-selection inference: interactive multiple testing, fixed-design linear regression, generalized linear models, and trend filtering.

24. A composite generalization of Ville's martingale theorem using e-processes (with J. Ruf, M. Larsson, W. Koolen), Elec. J. of Prob., 2023 arXiv | proc | TLDR

    Ville's famous martingale theorem relates measure-theoretic probability to betting: it states that for any event of measure zero, there exists a nonnegative martingale (a betting strategy) that multiplies its initial wealth infinitely if the event occurs. We prove a composite generalization of that theorem, which requires generalizing ``measure zero'' to a certain inverse capital outer measures and generalizing ``nonnegative martingale'' to e-processes.

25. Online multiple hypothesis testing (with D. Robertson, J. Wason), Statistical Science, 2023 arXiv | proc

26. Nonparametric two-sample testing by betting (with S. Shekhar), IEEE Trans. on Info. Theory, 2023       arXiv | proc | code | slides | TLDR

    We develop a general framework for designing sequential two-sample tests, and obtain a general characterization of the power of these tests in terms of the regret of an associated online prediction game. This yields the ``right'' sequential generalizations of many offline nonparametric two-sample tests like Kolmogorov-Smirnov or kernel-MMD.

27. E-values as unnormalized weights in multiple testing (with N. Ignatiadis, R. Wang), Biometrika, 2023 arXiv | proc

28. Comparing sequential forecasters (with Y.J. Choe), Operations Research, 2023 arXiv | proc | code | talk | poster | slides (Citadel, Research Showcase Runner-up)

29. Game-theoretic statistics and safe anytime-valid inference (with P. Grunwald, V. Vovk, G. Shafer), Statistical Science, 2023 arXiv | proc

30. Adaptive privacy composition for accuracy-first mechanisms (with R. Rogers, G. Samorodnitsky, S. Wu), Conf. on Neural Information Processing Systems (NeurIPS), 2023 arXiv | proc | TLDR

    We derive basic and advanced composition results and privacy filters for noise-reduction mechanisms that allow an analyst to adaptively switch between differentially private and ex-post private mechanisms subject to an overall privacy guarantee.

31. Sequential predictive two-sample and independence testing (with A. Podkopaev), Conf. on Neural Information Processing Systems (NeurIPS), 2023 arXiv | proc

32. Auditing fairness by betting (with B. Chugg, S. Cortes-Gomez, B. Wilder), Conf. on Neural Information Processing Systems (NeurIPS), 2023 arXiv | code | proc

33. Counterfactually comparing abstaining classifiers (with Y. J. Choe, A. Gangrade), Conf. on Neural Information Processing Systems (NeurIPS), 2023 arXiv | slides | proc

34. An efficient doubly-robust test for the kernel treatment effect (with D. Taboada, E. Kennedy), Conf. on Neural Information Processing Systems (NeurIPS), 2023 arXiv | proc

35. On the sublinear regret of GP-UCB (with J. Whitehouse, S. Wu), Conf. on Neural Information Processing Systems (NeurIPS), 2023 arXiv | TLDR

    By appropriately regularizing simple confidence sequences in Hilbert spaces, we derive (for the first time) sublinear regret for GP-UCB for any kernel with polynomial decay (including Matern).

36. Martingale methods for sequential estimation of convex functionals and divergences (with T. Manole), IEEE Trans. on Information Theory, 2023 arXiv | article | talk (Student Research Award, Stat Soc Canada) | TLDR

    We derive confidence sequences for convex functionals, with an emphasis on convex divergences such as the kernel Maximum Mean Discrepancy and Wasserstein distances; our main technical contribution is to show that empirical plugins of convex functionals/divergences (and more generally processes satisfying a leave-one-out property) are partially ordered reverse submartingales, coupled with maximal inequalities for such processes.

37. Estimating means of bounded random variables by betting (with I. Waudby-Smith), J. of the Royal Statistical Society, Series B, 2023 arXiv (Discussion paper) | proc | code

38. Sequential change detection via backward confidence sequences (with S. Shekhar). Intl. Conf. on Machine Learning (ICML), 2023   arXiv | code | slides | TLDR

    We derive a general reduction from constructing confidence sequences (CSs) for some functional~(say $\theta$) to detecting changes in that functional: given a stream of observations, construct a single CS (for $\theta$) in the forward direction, and a new CS in the backward direction with each new observation, and stop and declare a changepoint as soon as they do not intersect. We obtain tight guarantees on the average run length~(ARL) and the detection delay for this general strategy, and instantiate them for several classical and modern change detection problems.

39. Fully adaptive composition in differential privacy (with J. Whitehouse, R. Rogers, Z. S. Wu), Intl. Conf. on Machine Learning (ICML), 2023 arXiv | proc

40. Online Platt scaling with calibeating (with C. Gupta), Intl. Conf. on Machine Learning (ICML), 2023 arXiv | proc

41. A nonparametric extension of randomized response for locally private confidence sets (with I. Waudby-Smith, Z. S. Wu), Intl. Conf. on Machine Learning (ICML), 2023 arXiv | code (oral talk)

42. Sequential kernelized independence testing (with A. Podkopaev, P. Bloebaum, S. Kasiviswanathan), Intl. Conf. on Machine Learning (ICML), 2023 arXiv | proc | code

43. Risk-limiting financial audits via weighted sampling without replacement (with S. Shekhar, Z. Xu, Z. Lipton, P. Liang), Intl. Conf. Uncertainty in AI (UAI), 2023 arXiv | proc TLDR

    We introduce the notion of risk-limiting financial audits (RLFA), where the goal is to design statistical procedures to verify an assertion about a set of reported financial transactions. We propose a general RLFA strategy using confidence sequences constructed with weighted sampling without replacement, and also develop techniques that can incorporate any available side information (such as predictions from AI models).

44. Huber-robust confidence sequences (with H. Wang), Intl. Conf. on AI and Statistics (AISTATS), 2023, arXiv (full oral talk) | TLDR

    Under a slight generalization of Huber's epsilon-contamination model (where epsilon fraction of the points are arbitrarily corrupted), we derive confidence sequences for univariate means only assuming a finite p-th moment (for p between 1 and 2), which are minimax optimal and perform very well in practice.

45. Catoni-style confidence sequences for heavy-tailed mean estimation (with H. Wang), Stochastic Processes and Applications, 2023 arXiv | article | code | TLDR

    We derive confidence sequences, which are confidence intervals valid at arbitrary stopping times, for univariate means only assuming a finite p-th moment (for p between 1 and 2), which are minimax optimal and perform very well in practice.

46. Anytime-valid confidence sequences in an enterprise A/B testing platform (with A. Maharaj, R. Sinha, D. Arbour, I. Waudby-Smith, S. Liu, M. Sinha, R. Addanki, M. Garg, V. Swaminathan), ACM Web Conference (WWW), 2023 arXiv

47. Dimension-agnostic inference using cross U-statistics (with I. Kim), Bernoulli, 2023 arXiv | proc | TLDR

    We introduce dimension-agnostic inference, which is a novel approach for high-dimensional inference that ensures asymptotic validity regardless of how the dimension and sample size scale, while preserving minimax optimal power across diverse scenarios; our main tool is a cross U-statistic, which drops half of the terms of a degenerate U-statistic to yield a limiting Gaussian distribution.

48. On the power of conditional independence testing under model-X (with E. Katsevich), Electronic J. Stat, 2023 arXiv | article

49. Permutation tests using arbitrary permutation distributions (with R. Barber, E. Candes, R. Tibshirani), Sankhya A, 2023 arXiv | article

50. Conformal prediction beyond exchangeability (with R. Barber, E. Candes, R. Tibshirani), Annals of Stat., 2023 arXiv | article

51. Faster online calibration without randomization: interval forecasts and the power of two choices (with C. Gupta), Conf. on Learning Theory (COLT), 2022 arXiv | article

52. Top-label calibration and multiclass-to-binary reductions (with C. Gupta), Intl. Conf. on Learning Representations, 2022 arXiv | article

53. Gaussian universal likelihood ratio testing (with R. Dunn, S. Balakrishnan, L. Wasserman), Biometrika, 2022 arXiv | article | TLDR

    Under a Gaussian setting, we present the first in-depth exploration of the size, power, and relationships between several universal inference variants. We find that in this setting, the power of universal inference has the same behavior in n, d, alpha and SNR as the classical Wilks' Theorem approach, only losing in a small constant of about 2.

54. A permutation-free kernel two sample test (with S. Shekhar, I. Kim), Conf. on Neural Information Processing Systems (NeurIPS), 2022 arXiv | article | code | (oral talk) | TLDR

    We propose a new kernel-MMD statistic that drops half the terms of the original, and show that it has a standard normal limiting null distribution in low and high dimensional regimes. This results in a test that is easy to calibrate, that is up to two orders of magnitude faster than running the permutation test, at the price of a small ($\approx \sqrt{2}$ in effective sample size) reduction in power.

55. Testing exchangeability: fork-convexity, supermartingales, and e-processes (with J. Ruf, M. Larsson, W. Koolen). Intl J. of Approximate Reasoning, 2022 arXiv | article

56. Tracking the risk of a deployed model and detecting harmful distribution shifts (with A. Podkopaev). Intl. Conf. on Learning Representations (ICLR), 2022 arXiv | article

57. Brownian noise reduction: maximizing privacy subject to accuracy constraints (with J. Whitehouse, Z.S. Wu, R. Rogers), Conf. on Neural Information Processing Systems (NeurIPS), 2022 arXiv | article

58. Sequential estimation of quantiles with applications to A/B-testing and best-arm identification (with S. Howard), Bernoulli, 2022 arXiv | article | code

59. Brainprints: identifying individuals from magnetoencephalograms (with S. Wu, L. Wehbe), Nature Communications Biology, 2022 bioRxiv | article

60. Interactive rank testing by betting (with B. Duan, L. Wasserman), Conf. on Causal Learning and Reasoning (CLEAR), 2022 arXiv | article (oral talk)

61. Large-scale simultaneous inference under dependence (with J. Tian, X. Chen, E. Katsevich, J. Goeman), Scandanavian J of Stat., 2022 arXiv | article

62. False discovery rate control with e-values (with R. Wang), J. of the Royal Stat. Soc., Series B, 2022 arXiv | article

63. Nested conformal prediction and quantile out-of-bag ensemble methods (with C. Gupta, A. Kuchibhotla), Pattern Recognition, 2022 arXiv | article | code

64. Distribution-free prediction sets for two-layer hierarchical models (with R. Dunn, L. Wasserman), J of American Stat. Assoc., 2022 arXiv | article | code | TLDR

    Conformal methods typically rely on exchangeable data to provide valid prediction sets in finite samples, but we extend conformal methods to construct prediction sets in a nonexchangeable two-layer hierarchical setting, where N groups of data are exchangeable, and the observations within each group are also exchangeable.

65. Fast and powerful conditional randomization testing via distillation (with M. Liu, E. Katsevich, L. Janson), Biometrika, 2021 arXiv | article | code

66. Uncertainty quantification using martingales for misspecified Gaussian processes (with W. Neiswanger), Algorithmic Learning Theory (ALT), 2021 arXiv | article | code | talk

67. RiLACS: Risk-limiting audits via confidence sequences (with I. Waudby-Smith, P. Stark), Intl. Conf. for Electronic Voting (EVoteID), 2021 arXiv | article | code (Best Paper award)

68. Predictive inference with the jackknife+ (with R. Barber, E. Candes, R. Tibshirani), Annals of Stat., 2021 arXiv | article | code

69. Path length bounds for gradient descent and flow (with C. Gupta, S. Balakrishnan), J. of Machine Learning Research, 2021 arXiv | article | blog

70. Nonparametric iterated-logarithm extensions of the sequential generalized likelihood ratio test (with J. Shin, A. Rinaldo), IEEE J. on Selected Areas in Info. Theory, 2021 arXiv | article

71. Time-uniform, nonparametric, nonasymptotic confidence sequences (with S. Howard, J. Sekhon, J. McAuliffe), The Annals of Stat., 2021 arXiv | article | code | tutorial

72. Off-policy confidence sequences (with N. Karampatziakis, P. Mineiro), Intl. Conf. on Machine Learning (ICML), 2021 arXiv | article

73. Best arm identification under additive transfer bandits (with O. Neopane, A. Singh), Asilomar Conf. on Signals, Systems and Computers, 2021 arXiv | article (Best Student Paper award)

74. On the bias, risk and consistency of sample means in multi-armed bandits (with J. Shin, A. Rinaldo), SIAM J. on the Math. of Data Science, 2021 arXiv | article | talk

75. Dynamic algorithms for online multiple testing (with Z. Xu), Conf. on Math. and Scientific Machine Learning, 2021 arXiv | article | talk | slides | code | TLDR

    We develop the first practically powerful algorithms that provably controls the supremum of the false discovery proportion with high probability in online multiple testing.

76. Online control of the familywise error rate (with J. Tian), Statistical Methods in Medical Research, 2021 arXiv | article

77. Asynchronous online testing of multiple hypotheses (with T. Zrnic, M. Jordan), J. of Machine Learning Research, 2021 arXiv | article | code | blog

78. Classification accuracy as a proxy for two sample testing (with I. Kim, A. Singh, L. Wasserman), Annals of Stat., 2021 arXiv | article | (JSM Stat Learning Student Paper Award) | TLDR

    We explore the use of classification accuracy for two-sample testing for general classifiers, proving in particular that the accuracy test based on Fisher's LDA achieves minimax rate-optimal power and establishing conditions for consistency based on general classifiers.

79. Distribution-free calibration guarantees for histogram binning without sample splitting (with C. Gupta), Intl. Conf. on Machine Learning, 2021 arXiv | article

80. Distribution-free uncertainty quantification for classification under label shift (with A. Podkopaev), Conf. on Uncertainty in AI, 2021 arXiv | article

81. Distribution-free binary classification: prediction sets, confidence intervals and calibration (with C. Gupta, A. Podkopaev), Conf. on Neural Information Processing Systems (NeurIPS), 2020 arXiv | article (spotlight talk)

82. The limits of distribution-free conditional predictive inference (with R. Barber, E. Candes, R. Tibshirani), Information and Inference, 2020 arXiv | article

83. Analyzing student strategies in blended courses using clickstream data (with N. Akpinar, U. Acar), Educational Data Mining, 2020 arXiv | article | talk (oral talk)

84. The power of batching in multiple hypothesis testing (with T. Zrnic, D. Jiang, M. Jordan), Intl. Conf. on AI and Statistics, 2020 arXiv | article | talk

85. Online control of the false coverage rate and false sign rate (with A. Weinstein), Intl. Conf. on Machine Learning (ICML), 2020 arXiv | article

86. Confidence sequences for sampling without replacement (with I. Waudby-Smith), Conf. on Neural Information Processing Systems (NeurIPS), 2020 arXiv | article | code (spotlight talk)

87. Universal inference (with L. Wasserman, S. Balakrishnan), Proc. of the National Academy of Sciences, 2020 arXiv | article | talk

88. A unified framework for bandit multiple testing (with Z. Xu, R. Wang), Conf. on Neural Information Processing Systems (NeurIPS), 2020 arXiv | article | talk | slides | code | TLDR

    Using e-values (or e-processes) and the e-BH procedure, we formulate a framework which provides false discovery rate (FDR) control at any stopping time for multiple testing in the bandit setting, that is robust to the dependencies induced by the user’s sampling and stopping policies.

89. Simultaneous high-probability bounds on the FDP in structured, regression and online settings (with E. Katsevich), Annals of Stat., 2020 arXiv | article | code

90. Time-uniform Chernoff bounds via nonnegative supermartingales (with S. Howard, J. Sekhon, J. McAuliffe), Prob. Surveys, 2020 arXiv | article | talk

91. STAR: A general interactive framework for FDR control under structural constraints (with L. Lei, W. Fithian), Biometrika, 2020 arXiv | article | poster | code

92. Familywise error rate control by interactive unmasking (with B. Duan, L. Wasserman), Intl. Conf. on Machine Learning (ICML), 2020 arXiv | article | code

93. Interactive martingale tests for the global null (with B. Duan, S. Balakrishnan, L. Wasserman), Electronic J. of Stat., 2020 arXiv | article | code

94. On conditional versus marginal bias in multi-armed bandits (with J. Shin, A. Rinaldo), Intl. Conf. on Machine Learning (ICML), 2020 arXiv | article

95. Are sample means in multi-armed bandits positively or negatively biased? (with J. Shin, A. Rinaldo), Conf. on Neural Information Processing Systems (NeurIPS), 2019 arXiv | article | poster

96. A higher order Kolmogorov-Smirnov test (with V. Sadhanala, Y. Wang, R. Tibshirani), Intl. Conf. on AI and Statistics, 2019 arXiv | article

97. ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls (with J. Tian), Conf. on Neural Information Processing Systems (NeurIPS), 2019 arXiv | code | article

98. A unified treatment of multiple testing with prior knowledge using the p-filter (with R. F. Barber, M. Wainwright, M. Jordan), Annals of Stat., 2019 arXiv | article | code

99. DAGGER: A sequential algorithm for FDR control on DAGs (with J. Chen, M. Wainwright, M. Jordan), Biometrika, 2019 arXiv | article | code

100. Conformal prediction under covariate shift (with R. Tibshirani, R. Barber, E. Candes), Conf. on Neural Information Processing Systems (NeurIPS), 2019 arXiv | article | poster

101. Optimal rates and tradeoffs in multiple testing (with M. Rabinovich, M. Wainwright, M. Jordan), Statistica Sinica, 2019 arXiv | article | poster

102. Function-specific mixing times and concentration away from equilibrium (with M. Rabinovich, M. Wainwright, M. Jordan), Bayesian Analysis, 2019 arXiv | article | poster

103. Decoding from pooled data (II): sharp information-theoretic bounds (with A. El-Alaoui, F. Krzakala, L. Zdeborova, M. Jordan), SIAM J. on Math. of Data Science, 2019 arXiv | article

104. Decoding from pooled data (I): phase transitions of message passing (with A. El-Alaoui, A. Ramdas, F. Krzakala, L. Zdeborova, M. Jordan), IEEE Trans. on Info. Theory, 2018 arXiv | article

105. On the power of online thinning in reducing discrepancy (with R. Dwivedi, O. N. Feldheim, Ori Gurel-Gurevich), Prob. Theory and Related Fields, 2018 arXiv | article | poster

106. On kernel methods for covariates that are rankings (with H. Mania, M. Wainwright, M. Jordan, B. Recht), Electronic J. of Stat., 2018 arXiv | article

107. SAFFRON: an adaptive algorithm for online FDR control (with T. Zrnic, M. Wainwright, M. Jordan), Intl. Conf. on Machine Learning (ICML), 2018 arXiv | article | code (full oral talk)

108. Online control of the false discovery rate with decaying memory (with F. Yang, M. Wainwright, M. Jordan), Conf. on Neural Information Processing Systems (NeurIPS), 2017 arXiv | article | poster | talk (from 44:00) (full oral talk)

109. MAB-FDR: Multi (A)rmed\/(B)andit testing with online FDR control (with F. Yang, K. Jamieson, M. Wainwright), Conf. on Neural Information Processing Systems (NeurIPS), 2017 arXiv | article | code (spotlight talk)

110. QuTE: decentralized FDR control on sensor networks (with J. Chen, M. Wainwright, M. Jordan), IEEE Conf. on Decision and Control, 2017 arXiv | article | code | poster

111. Iterative methods for solving factorized linear systems (with A. Ma, D. Needell), SIAM J. on Matrix Analysis and Applications, 2017 arXiv | article

112. Rows vs. columns : randomized Kaczmarz or Gauss-Seidel for ridge regression (with A. Hefny, D. Needell), SIAM J. on Scientific Computing, 2017 arXiv | article

113. On Wasserstein two sample testing and related families of nonparametric tests (with N. Garcia, M. Cuturi), Entropy, 2017 arXiv | article

114. Generative models and model criticism via optimized maximum mean discrepancy (with D. Sutherland, H. Tung, H. Strathmann, S. De, A. Smola, A. Gretton), Intl. Conf. on Learning Representations, 2017 arXiv | article | poster | code

115. Minimax lower bounds for linear independence testing (with D. Isenberg, A. Singh, L. Wasserman), IEEE Intl. Symp. on Information Theory, 2016 arXiv | article

116. p-filter: multi-layer FDR control for grouped hypotheses (with COAUTHORS), J. of the Royal Stat. Society, Series B, 2016 arXiv | article |code | poster

117. Sequential nonparametric testing with the law of the iterated logarithm (with A. Balsubramani), Conf. on Uncertainty in AI, 2016 arXiv | article | errata

118. Asymptotic behavior of Lq-based Laplacian regularization in semi-supervised learning (with A. El-Alaoui, X. Cheng, M. Wainwright, M. Jordan), Conf. on Learning Theory, 2016 arXiv | article

119. Regularized brain reading with shrinkage and smoothing (with L. Wehbe, R. Steorts, C. Shalizi), Annals of Applied Stat., 2015 arXiv | article

120. On the high-dimensional power of a linear-time two sample test under mean-shift alternatives (with S. Reddi, A. Singh, B. Poczos, L. Wasserman), Intl. Conf. on AI and Statistics, 2015 arXiv | article | errata

121. On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions (with S. Reddi*, B. Poczos, A. Singh, L. Wasserman), AAAI Conf. on Artificial Intelligence, 2015 arXiv | article | supp

122. Fast two-sample testing with analytic representations of probability measures (with K. Chwialkowski, D. Sejdinovic, A. Gretton), Conf. on Neural Information Processing Systems (NeurIPS), 2015 arXiv | article | code

123. Nonparametric independence testing for small sample sizes (with L. Wehbe), Intl. Joint Conf. on AI, 2015 arXiv | article (oral talk)

124. Convergence properties of the randomized extended Gauss-Seidel and Kaczmarz methods (with A. Ma, D. Needell), SIAM J. on Matrix Analysis and Applications, 2015 arXiv | article | code

125. Fast & flexible ADMM algorithms for trend filtering (with R. Tibshirani), J. of Computational and Graphical Statistics, 2015 arXiv | article | talk | code

126. Towards a deeper geometric, analytic and algorithmic understanding of margins (with J. Pena), Opt. Methods and Software, 2015 arXiv | article

127. Margins, kernels and non-linear smoothed perceptrons (with J. Pena), Intl. Conf. on Machine Learning (ICML), 2014 arXiv | article | poster | talk (oral talk)

128. Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses (with L. Wehbe, B. Murphy, P. Talukdar, A. Fyshe, T. Mitchell), PLoS ONE, 2014 website | article

129. An analysis of active learning with uniform feature noise (with A. Singh, L. Wasserman, B. Poczos), Intl. Conf. on AI and Statistics, 2014 arXiv | article | poster | talk (oral talk)

130. Algorithmic connections between active learning and stochastic convex optimization (with A. Singh), Conf. on Algorithmic Learning Theory (ALT), 2013 arXiv | article | poster

131. Optimal rates for stochastic convex optimization under Tsybakov's noise condition (with A. Singh), Intl. Conf. on Machine Learning (ICML), 2013 arXiv | article | poster | talk (oral talk)

Miscellaneous

1. Adaptivity & computation-statistics tradeoffs for kernel & distance based high-dimensional two sample testing (with S. Reddi, B. Poczos, A. Singh, L. Wasserman).       arXiv | poster

2. Algorithms for graph similarity and subgraph matching (with D. Koutra, A. Parikh, J. Xiang).       report